In FY18-19, the iOS team will develop contribution features of the app (project page).

This is an initial exploration with two goals:

  • Write relevant queries that can be productionized later (made to execute daily with data flowing into Druid/Superset).
  • Learn what the program metrics look like right now so we can set our expectations accordingly.

Metrics

Primary metrics

  • Total iOS app edit counts: Number of non-bot edits which are tagged as made through the iOS app interface.
  • iOS edits per editor: For any given time period (monthly/quarterly), of the non-bot registered users that completed at least 1 edit via the iOS app, how many edits did they complete on average (arithmetic mean/median). Edits on other platform won’t affect this metric.

Health monitoring metrics

  • Revert rate of iOS edits: Proportion of “ios app edit”-tagged contributions which have been reverted
  • iOS active contributor retention rate: Number of active iOS editors who have made additional edits through the iOS app in the 30 days following their first iOS edit in the given time span (does not include editors who have only made additional edits through other platforms following their first iOS edit)
  • iOS new contributor retention rate: Number of new iOS editors who have made additional edits through the iOS app in the 30 days following their first ever iOS edit (does not include editors who have only made additional edits through other platforms following their first iOS edit)

In this report, we will compute the baseline for all languages of Wikipedia, with a focus on our target wikis in the annual plan: French, English, Korean, Hindi and Czech.

Note: Platform-specific edit data was not available until June 29 via T194424. Prior to the deployment of the patch, all edits made in iOS and Android apps were lumped together under a single change tag “mobile app edit”. Since multi-platform usage and platform switching are not common occurrences, any editor who has made a “mobile app edit”-tagged contribution previously is ineligble to be considered a new iOS editor.

Total iOS app edit counts

Number of non-bot edits which are tagged as made through the iOS app interface. Bot filter are not included in the query since it’s very unlikely that a bot would use the app.

Query:

SELECT DATE(date) AS date,
IFNULL(SUM(edits), 0) AS edits,
IFNULL(SUM(content_edits), 0) AS content_edits,
IFNULL(SUM(deleted_edits), 0) AS deleted_edits
FROM (
SELECT
LEFT(rev_timestamp, 8) AS `date`,
COUNT(*) AS `edits`,
SUM(page_namespace = 0) AS content_edits,
SUM(rev_deleted = 1) AS deleted_edits
FROM revision
INNER JOIN change_tag ON rev_id = ct_rev_id AND ct_tag = 'ios app edit'
LEFT JOIN page ON rev_page = page_id
WHERE rev_timestamp >= '{start_date}'
AND rev_timestamp < '{end_date}'
GROUP BY LEFT(rev_timestamp, 8)

UNION ALL

SELECT
LEFT(ar_timestamp, 8) AS `date`,
COUNT(*) AS `edits`,
SUM(ar_namespace = 0) AS content_edits,
COUNT(*) AS deleted_edits
FROM archive
INNER JOIN change_tag ON ar_rev_id = ct_rev_id AND ct_tag = 'ios app edit'
WHERE ar_timestamp >= '{start_date}'
AND ar_timestamp < '{end_date}'
GROUP BY LEFT(ar_timestamp, 8)
) AS edit_counts
GROUP BY date

Target Wikipedia

All Edits


Content Edits

Edit counts in the 0 namespace.


All Active Wikipedia

All Edits


Content Edits

Edit counts in the 0 namespace.


iOS edits by editor

For any given time period (monthly/quarterly), of the non-bot registered users that completed at least 1 edit via the iOS app, how many edits did they complete on average (arithmetic mean/median). Edits on other platform won’t affect this metric. Bot filter are not included since it’s very unlikely that a bot would use the app.

Here we started by counting the number of edits per editor in July 2018 without doing any aggregation, since we want to see the distribution before determine whether we should use mean or median, or other aggregation metrics.

Query:

SELECT month,
local_user_id,
IFNULL(user_name, '') AS user_name,
IFNULL(SUM(edits), 0) AS edits,
IFNULL(SUM(content_edits), 0) AS content_edits,
IFNULL(SUM(deleted_edits), 0) AS deleted_edits
FROM (

SELECT
LEFT(rev_timestamp, 6) AS `month`,
rev_user AS `local_user_id`,
COUNT(*) AS `edits`,
SUM(page_namespace = 0) AS content_edits,
SUM(rev_deleted = 1) AS deleted_edits
FROM revision
INNER JOIN change_tag ON rev_id = ct_rev_id AND ct_tag = 'ios app edit'
LEFT JOIN page ON rev_page = page_id
WHERE LEFT(rev_timestamp, 6) = '201807'
GROUP BY LEFT(rev_timestamp, 6), rev_user

UNION ALL

SELECT
LEFT(ar_timestamp, 6) AS `month`,
ar_user AS `local_user_id`,
COUNT(*) AS `edits`,
SUM(ar_namespace = 0) AS content_edits,
COUNT(*) AS deleted_edits
FROM archive
INNER JOIN change_tag ON ar_rev_id = ct_rev_id AND ct_tag = 'ios app edit'
WHERE LEFT(ar_timestamp, 6) = '201807'
GROUP BY LEFT(ar_timestamp, 6), ar_user

) AS edit_per_editor
LEFT JOIN user ON local_user_id = user_id
GROUP BY month, local_user_id

Target Wikipedia

All Edits



Content Edits



All Active Wikipedia

All Edits


Content Edits


Plots

Suggestion

From the tables and plots above, the distribution of edits per editor is very right skewed with lots of 1-edit editors, thus we can see the average or median edits per editor are not very descriptive. Therefore, I suggest we:

  • Track the proportion of 1-edit editors among all registered editors who edit at least once via iOS app for any given time period (monthly/quarterly). The smaller this number is, the better.
  • Since we consider editors with at least 5 edits per month as active editors (this threshold normally applies to desktop edits, so we should probably find another threshold for mobile editing), we want to track the proportion of 5-edit editors among all registered editors who edit at least once via iOS app monthly. The larger this number is, the better.
  • Since the iOS team’s goal is to improve the experience for existing editors, for “iOS heavy wikis” (we still need to define it clearly), we may want to track the number of editors who edit at least once via iOS app for any given time period (monthly/quarterly).